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INTRODUCTION 

The Jet Propulsion Laboratory (JPL) and the 
Electrotechnical Laboratory (ETL) have 
recently initiated a cooperative R&D effort in 
telerobotics. This new effort, sponsored by the 
U.S. National Aeronautics and Space 
Administration (NASA) and Japan’s Ministry of 
International Trade and Industry (MITI), has 
two major themes. First, our work broadens the 
outreach of space telerobotics R&D to 
international technical collaboration and 
facilities usage in the United States and Japan. 
This is natural, given plans for a common U.S.- 
Japan robotic presence on the International 
Space Station (the Japanese Experimental 
Module and U.S. Mobile Servicing Center), as 
well as ongoing U.S-Japan discussions of 
possible shared ground control assets. Second, 
our work fosters development and 
demonstration of new operator interface 
technologies to improve the flexibility and 
reliability of ground-to-orbit telerobotic 
operations. This new technology is important, 
given the continuing imperatives to off-load 
platform maintenance from the extravehicular 
activity/intravehicular activity (EVA/IVA) crew 
to on-board robot assists under direct ground 
mission control [1]. Permanent human 
capability and productive science on platforms 


such as the Space Station will otherwise be 
delayed. 

COMMON TRADITIONS, 
COMPLEMENTARY STRENGTHS AT 
ETL AND JPL 

JPL and ETL share a long-standing interest 
in human-computer cooperative control of 
robots, and its applications in casually structured 
tasks such as space assembly and servicing, 
hazardous materials handling, and telescience. 
Use of such supervised autonomy [2], versus 
total robotic automation, is necessitated because 
computer control of robots is not yet adequate to 
make complete task plans, learn tasks at the 
cognitive and motor skill of humans, or execute 
tasks with the dexterity of human servo-motor 
performance. At the other extreme, purely 
manual control of robots by teleoperation is 
often time-consuming and fatiguing, poorly 
suited to repeated actions of high precision, and 
impractical in scenarios where the operator's 
sensory feedback is significantly time-delayed. 
As regards the technologies they bring to the 
NASA-MITI collaboration, JPL and ETL have 
chosen complementary approaches to 
developing supervisory automation. JPL's 
approach, consistent with its space operations 
charter, derives from computer-augmented 
teleoperation [3, 4], the goal to date having been 
to maximize manual tasking dexterity and 
telepresence, and extend both to multiple- 
second time-delay remote-servicing scenarios. 
For example, JPL, utilizing its development of 
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a dual eight-degree-of-freedom force reflecting 
teleoperator with multi-mode controls (position, 
force, rate, compliance, shared computer control 
of some axes) has recently re-enacted various 
aspects of the Solar Maximum Satellite repair 
mission conducted on shuttle flight STS- 13; in 
this earth laboratory simulation, JPL 
telerobotically performed key sequences of this 
benchmark 1984 dual-EVA shuttle-bay 
servicing mission. Two JPL enabling 
technology developments have been calibrated 
preview/predictive graphics displays [4] and 
shared compliance control [3]. Using such a 
preview/predictive graphics operator interface 
and a related robot compliance control, JPL and 
NASA Goddard Space Flight Center (GSFC) 
recently performed simulated ground-to-orbit 
space telerobotic servicing under multiple- 
second variable communication time delay, 
wherein JPL successfully changed out an 
Orbital Replaceable Unit (ORU) of a Hubble 
Space Telescope-like spacecraft mock-up 
located some 4000 kilometers distant at GSFC 
[4]. 

By comparison to the above JPL work, ETL 
has recently emphasized higher level intelligent 
and cooperative control interactions between 
humans and robots [5, 6], Consistent with a 
strong interest in flexible assembly operations, 
ETL seeks to relieve operator fatigue through 
automation, yet allow the operator to manually 
interact with robot automation with ease if 
needed. For example, ETL has demonstrated 
intelligent and cooperative control in robotic 
chemical assay by flame test. The robot, under 
supervised autonomy, sets up, pulverizes, 
samples and flame-tests chemical substances, 
with the operator intervening to graphically re- 
designate locations of desired actions or 
teleoperate to deal with task anomalies. ETL has 
developed the MEISTER (Model Enhanced 
Intelligent and Skillful Teleoperational Robot) 
system architecture to enable such supervisory 
control [5]. A key design feature of this 
architecture is the embedding of environmental 
and control knowledge within a collection of 
task-oriented object models, wherein the model 
representation itself is "object-oriented," e.g., 
each object model contains self-knowledge such 
as position and orientation with respect to world 
coordinates ("object localization") and its 
affixment relationships to other objects. The 
object models embed both generic and specific 
handling knowledge, such that the commanding 
of a control operation, e.g., pick_and_ place, 


invokes a linked hierarchy of processes, 
including the automatic sequencing of basic 
camera- viewing primitives [6]. 

NEAR-TERM PLANS AND PROGRESS 

JPL and ETL separately fund their U.S.- 
Japan telerobotics R&D cooperation through 
projects respectively entitled "Distributed Space 
Telerobotics," and "Interoperation Technology 
for Long-Distance Robotics." These efforts, 
which independently develop their component 
technologies, converge in jointly implemented 
overseas system demonstrations. The first 
planned experiment (US-FY95) is truss-based 
telerobotic deployment of a solar-powered 
Orbital Replaceable Unit (ORU) and electrical 
connectors. This operation will be performed 
from JPL by a joint ETL-JPL team controlling 
an ETL robot. There will be a reciprocal 
operations experiment (US-FY96) from ETL to 
JPL where a joint JPL-ETL team will perform 
telerobotic servicing of a limited-access ORU in 
a simulated Space Station environment. In 
general, these experimental demonstrations and 
underlying technology developments highlight 
robust telerobotic operation under uncertainty. 
Major sources of operational uncertainty include 
effects of time delay, limited camera viewing, 
and lack of prior task knowledge. We are 
addressing two corresponding key technology 
needs [1], The first technology need is to 
develop an intelligent interface for operator 
visualization of complex workspaces , as 
motivated by the requirements to safely perform 
robotic servicing tasks in physically obstructed, 
limited viewing access structures, and also to 
maximize viewing automation under well- 
structured operating conditions. Desired 
capabilities include a computer planned-and- 
task-synchronized presentation of the global 
workspace that fuses remote multi-camera video 
with 3-D graphics, and correlates this display 
with operator information requirements for 
specific task processes and interventions. 
Measurable outcomes will include: a) reduction 
of the operations time used for manual camera 
control during a task, which often outweighs 
manipulation time, and usually requires an 
additional operator, b) the capability, through a 
coherently integrated presentation of real and 
synthesized task views, to safely operate in 
scenarios where camera viewing alone is 
inadequate. JPL refers to this work as 
Intelligent Viewing Control (IVC), which is 
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well-motivated by the limited camera resources 
and on-orbit time available for their use in 
future Space Shuttle/Space Station external 
robotic operations [1]. Other important IVC 
applications are areal surveillance, medical 
viewing, and flexible automation workcells. 

ETL and JPL both conduct related R&D, with 
ETL emphasizing object-based models for 
camera view planning. JPL, emphasizing the 
real-time integration of 3-D graphics and an AI- 
based view controller, carried out this year 
proof-of-concept robotics experiments with a 
first-cut IVC subsystem implementation. 

The second, complementary technology 
need is to develop a ground control interface for 
dexterous robotic tasking under extended 
(2-10 s) and intermittent time delays, as 
motivated by the requirement to safely 
telemanipulate in casually structured, and a 
priori less-well-modeled scenarios. The 
problems of teleoperation at time delays 
exceeding one second are well known [2], and 
the most recent predictive graphics-based 
approaches [4], per above, have as yet advanced 
reliable operations to one-to-four seconds’ 
delay for a priori well-modeled tasks. The 
desired new capabilities are to elevate the 
predictive graphics-and-compliance control 
paradigm [3] to a more flexible 
"teleprogrammed" form of supervisory control. 
In this new approach, the operator still manually 
inputs motions to a modeled task environment. 

However, rather than these continuous 
operator motions being sent directly to the 
robot, they are first parsed by computer to 
discrete low-level autonomous commands, 
which are then communicated to the remote site 
asynchronously. Once received, the commands 
are interpreted by the robot controller as simple 
guarded motion control primitives referenced to 
real-time robot sensor data. This approach 
enables introduction of intelligent, corrective 
robot behaviors to compensate for problems that 
the time-delayed operator cannot immediately 
address — we note some preliminary progress 
below. Measurable outcomes of this work will 
include: a) successful demonstrations of the 
teleprogrammed mode at time delays up to 10 
seconds for representative align, cut, grasp, 
insert and detach operations, b) application to 
situations where prior object model knowledge 
is of low quality (re: shape, position, 
orientation), requiring either significant 
qualitative control adaptation by the robot 
and/or on-line task model refinement by 


operator-interactive 3-D graphics acquisition- 
and-calibration. JPL refers to this work as 
Intelligent Motion Control (IMC), which is 
well-motivated by needs for more flexibly 
structured ground control of spacecraft 
EVA/robotic maintenance and telescience 
handling on the Space Station [1], 

ETL and JPL initiated experimental 
interactions and reciprocal engineering visits 
between our robotics laboratories in fall 1993. 

To date, ETL-JPL have performed several 
simple experiments to verify basic inter-lab 
operability. Also, JPL, working with the 
University of Pennsylvania, implemented and 
demonstrated important elements of an IMC 
subsystem. 

ETL->.TPL Remote Operations. JPL and ETL 

engineers, working together at ETL's Intelligent 
Interface Systems Lab, remotely commanded 
the guarded motion trajectory of an 8-degree-of- 
freedom JPL arm about the perimeter of a 
satellite ORU access panel door, simulating a 
proximity operations inspection (eye-in-hand 
camera). 

.TPL->ETL Remote Operations. JPL 

engineers commanded a robot at ETL in simple 
pick-and-place operations, via a high-level 
control interface. JPL sent the ETL robot 
control commands via a socket connection over 
Internet, from a LISP control program at JPL, to 
another LISP robot control program at ETL. 
Workspace models that enabled successful 
execution were resident within the robot control 
program at ETL. 

Intelligent Motion Control. JPL, working with 
University of Pennsylvania researchers [7] 
installed at JPL a real-time robot controller and 
command interfaces that compose the robot site 
of a "teleprogramming" facility, wherein the 
operator will command a time-and-space distant 
robot over communication links that may have 
variable delay. The fundamental University of 
Pennsylvania contribution in this work includes 
development at JPL of a novel layered 
behavioral control architecture. When active, 
this behavioral control replaces a more 
conventional hybrid position/force control, as 
conventionally used to correct quantitative 
variations in robot force and position along 
various axes of robot tool or gripper contact 
with an object of interest [3], and can 
autonomously compensate and strategically 
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correct for undesirable qualitative changes in the 
task state, as determined by the robot sensors. 
For example, the controller can assist a time- 
delayed operator in dealing with sudden, 
unpredictable disturbances and variations in 
contact with a workpiece being serviced, or 
object encountered. JPL-UPenn successfully 
demonstrated in January 1994 use of the 
behavioral controller to puncture and slice a 
Kapton tape seam securing satellite thermal 
blankets about a replica ORU main electronic 
box (MEB) access panel door. The controller 
successfully managed multiple, unpredictable 
metal-to-metal sidewall contacts as a cutting 
tool traveled laterally in a 2-mm-wide groove of 
a continuous 40-cm path sweep. Such tasks 
have challenged the skills of even experienced 
human operators in teleoperations tests. 
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